Co-evolution of dynamical states and interactions in dynamic networks 
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We explore the coupled dynamics of the internal states of a set of interacting elements and the 
network of interactions among them. Interactions are modeled by a spatial game and the network 
of interaction links evolves adapting to the outcome of the game. As an example we consider a 
model of cooperation, where the adaptation is shown to facilitate the formation of a hierarchical 
interaction network that sustains a highly cooperative stationary state. The resulting network 
has the characteristics of a small world network when a mechanism of local neighbor selection 
is introduced in the adaptive network dynamics. The highly connected nodes in the hierarchical 
structure of the network play a leading role in the stability of the network. Perturbations acting on 
the state of these special nodes trigger global avalanches leading to complete network reorganization. 
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Recent studies on the structure of social, technologi- 
cal and biological networks have shown that they share 
salient features which situate them far from being com- 
pletely regular or random 01 E 0- Most of the mod- 
els proposed to construct these networks are grounded 
in a graph-theoretical approach, i.e., algorithmic meth- 
ods to build graphs formed by elements (the nodes) and 
links that evolve according to pre-specified rules. Despite 
the progress made, there are still several open questions 
2J. An important issue to be considered among these 
questions is that networks are dynamical entities [J] that 
evolve and adapt driven by the actions of the elements 
that form a network. 

The aim of this paper is to analyze a simple setting 
of such adaptive and evolving network, in which there is 
co-evolution of the state of the elements in the nodes of 
the network and the interaction links defining the net- 
work. Interactions among elements are modeled with 
the aid of game theory frequently applied in social, 
economic and biological situations. This mathematical 
theory models an interaction involving two (or more) el- 
ements, each with two or more 'strategies' or states, such 
that the outcome depends on the choices of all the in- 
teracting elements. The outcome is given in the form 
of a 'utility' or payoff given to each element according 
to the selected action of the interacting elements. The 
introduction of spatial interactions lead to the develop- 
ment of 'spatial games' [HOUSi where the elements arc 
located in the nodes of a fixed network of interaction, dis- 
playing a rich spatio-temporal dynamics. We go here be- 
yond these studies by introducing adaptation {plasticity) 
in the coupling between elements, so that the network of 
interaction evolves adapting to the outcome of the game. 
Our results include new asymptotic steady states, and 
the emergence of a hierarchical network structure which 
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governs the global dynamics of the system. 

The model- We consider a system composed of N 
elements whose interactions are specified by a network 
M. The neighborhood of element i , Vi , is composed by 
those elements directly connected to i by one link, and 
the size of Vi defines its degree hi. The state of each 
element Xi can be (1,0) or (0,1). In each step (gen- 
eration), every «-th element interacts with all other ele- 
ments inside its neighborhood Vi , and accumulates a pay- 

off n, = v . 

eVi x i^ x j > depending on the chosen states 
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compares its own payoff with all j 6 Vi and changes its 
state to the state of the site with the greatest payoff in 
{i} U Vi 0- The plasticity of the network is introduced 
here as network dynamics in which existing links can be 
severed and replaced by new ones. We make the assump- 
tion that whether an interaction link is severed depends 
on the joint payoff, i.e. , the total payoff by the pair of 
interacting elements: the interactions giving the lowest 
benefit will be removed. 

In the remainder, for the sake of concreteness, we 
will address the case of the Prisoner's Dilemma (PD) 
game, which has been widely used as a model display- 
ing complex behavior emerging from the competition be- 
tween cooperative and selfish behavior |6| . In its simplest 
form, two elements may either choose to cooperate (C, 
xc — (1,0)), or defect (D, xjj = (0, 1)). If both elements 
choose C, each gets a payoff 7r o; if one defects while the 
other cooperates, the former gets payoff 7Tio > ttqq, while 
the latter gets the 'suckers' payoff 7r i < 7r o; if both 
defect, each gets ttu. Under the standard restrictions 
ttio + 7i"oi < 27T o, ttio > 7T o > 7Tn > 7t i, defection is the 
best choice in a one-shot game resulting in a Nash equi- 
librium where both elements defect. Following previous 
studies we consider a simplified version of the game 

given by the interaction matrix ttqq = 1, ttiq = 6,tth = e, 
ttqi = 0, in the limit e = [tj. 

In this context the dynamical rule proposed for local 
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FIG. 1: Average fraction of cooperative elements, pc, as a 
function of b and p in the stationary regime. The defective 
phase pc = is not included in the averages of pc- (p = full 
triangles, p = 0.01 circles, p = 0.1 squares, p = 1 diamonds) 




neighborhood adaptation, plasticity, is defined by ana- 
lyzing the joint benefit obtained by each of the possible 
pairwise interactions: C-C, C-D, and D-D. Thus, ac- 
cording to the payoff obtained the worst interaction is 
clearly observed in a D-D situation where both elements 
will be better off by searching a new partner. Given this 
simplistic analysis, taking into account that we are con- 
sidering undirected links, and assuming that the proba- 
bility to rewire a C-D interaction is much smaller than 
to rewire a D-D interaction, our implementation of plas- 
ticity will allow Defectors to exchange (probabilistically) 
a D-neighbor by another randomly chosen element. 

Thus, the game is divided in three stages, (i) Each 
element i plays the PD game with the same current state 
with all its neighbors, and collects an aggregate payoff II;. 
(ii) Each element i updates its current state by comparing 
its payoff with its neighbors and imitates the state of the 
wealthiest element. An element is said satisfied if its own 
payoff is the highest among its neighbors (otherwise it is 
unsatisfied), (iii) Unsatisfied D-elements which imitate 
a Defector, replace this link with probability p by a new 
one pointing to a randomly chosen element. 

The plasticity parameter p leads to a time evolution of 
the local connectivity of the network leaving the average 
degree (fej) constant. The parameter p sets a timescale 
for the evolution of the network with respect to the state 
update. In general we expect p <C 1, so that the state up- 
date evolves in a much faster timescale than the network 
evolution, while p — 1 represents the limit of simultane- 
ous update of interactions and states. 

We have characterized numerically the model using 
N = 10, 000 elements, averaged over 100 different ran- 
dom initial conditions, with an initial population of 0.6N 
Cooperators randomly distributed in the network 
The initial network is generated by randomly distributing 
N(ki)/2 links. A prototype value of (hi) = 8 was chosen 
in order to secure an initial large connected component. 
The game is played synchronously, i.e. , elements decide 
their state in advance and they all play at the same time. 

Stationary states. - To characterize the macroscopic be- 



FIG. 2: Partial view of a sample imitation network in a 
steady state. Elements on a lower layer imitate the state of 
elements in an upper layer. 



havior of the system we introduce the fraction of cooper- 
ators at a given time, pc{t)- We define the order param- 
eter pc as the average over realizations of the stationary 
cooperators density. In the case of random mixing, i.e. 
in the absence of an interaction network, population dy- 
namics gives pc = Pc(l — Pc)(l — b) . Thus, for 
b > 1 the only stable solution corresponds to a fully de- 
fective population. For fixed networks (p = 0), a typical 
time evolution shows in general that the order parame- 
ter fluctuates around an average value that decreases as 
the incentive to defect b increas es ( Fig. ^| . At b ~ 2 the 
defectors dominate the network [TJ- For fixed networks, 
the precise value for this transition has been studied in 
detail [3, 1 1 12L . In contrast to random mixing, context 
preservation (fixed interactions) sustains partial cooper- 
ation [l4|. 

This picture changes when the elements turn on their 
plasticity behavior (p > 0) [see Fig. [J]. Extensive nu- 
merical simulations show that the system either reaches 
a stationary configuration with pc > (where the states 
and the network do not change in time), or an absorbing 
state with all elements being Defectors pc = 0. The co- 
operative phase -the stationary states with a large value 
of pc~ is formed by a set of solutions, corresponding to 
different network configurations and distribution of co- 
operators. In Fig. we characterize these states showing 
that pc > 0.8, a value always much larger than in the 
non-adaptive case. Slight variations exist for different 
(k) > 4. The crucial difference is the disappearance of 
the behavior observed in the case p = in which, in- 
creasing b, the large majority of the realizations reaches 
a configuration with a very low fraction of cooperators. 
The plasticity parameter p changes the time it takes to 
reach the stationary state: smaller p produce longer tran- 
sients. 

Network structure.- In order to understand how such 
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FIG. 3: Normalized variance a\ = ({fef) — (ki) 2 )/(ki) as a 
function of b. The solid line (er£ = 1) correspond to the 
fixed random network with a Poisson distribution of degree. 
Parameter values as in Fig. Q 
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FIG. 4: Normalized clustering coefficient c/c ran d as a func- 
tion of b. For p = we recover the random value. Open 
symbols for q=0.01 (diamonds p = 0.01; squares p — 0.1; 
circles p = 1); filled circles correspond to p = 1 and q = 0. 



a highly cooperative structure can be sustained, we ana- 
lyze the implications of the proposed dynamical rules in 
the network structure. Consider that element i updates 
its state imitating the state of element j; we define the 
correspondence I : N — > TV such that l(i) — j. Focusing 
only on those links, we identify the imitation network 
as the sub-network composed of directed links i — » l(i) 
(Fig. |2J- A necessary and sufficient condition for a sta- 
tionary state (pc > 0, p ^ 0) is (a) there are no links 
between two Defectors, and (b) each C-neighbor i of a 
Defector 7, satisfies the payoff relation: 

ID, > n 7 > ID, j = (I) 

In other words, in a stationary state all defectors become 
satisfied interacting only with cooperators, while coop- 
erators can be unsatisfied while imitating from other co- 
operators. These steady state conditions naturally imply 
that the element with largest payoff in a stationary con- 
figuration is a satisfied cooperator. In Fig. [21 we show a 
partial view (the nodes in the lowest level are not shown) 
of an imitation network, where the nodes in a layer im- 
itate those elements in an upper layer indicated by the 
directed edges. At the top of the figure lie the nodes 
whose action is imitated by a chain of Cooperators. 

A first characterization of how the structure of the co- 
operative stationary network configurations changes as a 
function of b and p is obtained by measuring the normal- 
ized degree variance o 2 n — ((fcf) - (ki) 2 )/(ki) (Fig. |3J. 
We find that the degree distribution departs significantly 
from the initial Poisson distribution (cr„ = 1) only for 
large values of the plasticity parameter p. For increas- 
ing b the tail of the degree distribution expands and ap- 
proaches an exponential form, indicating some elements 
become more connected than others (hubs). 

We now address the question of whether the structure 
generated in our dynamical model has the characteristics 
of a small world network The clustering coefficient 
c measures the fraction of neighbors of a node that are 
connected among them, averaged over all the nodes in 
the network. In our simulations we find (Fig. that the 
clustering coefficient increases very mildly with respect 
to a fixed random network c ran d = (ki)/N |l5j . Thus, 



even though the average path length is similar to a ran- 
dom network, in order to account for the high clustering 
we need to introduce "local" neighbor selection [Tj?] . This 
mechanism is easily implemented introducing a parame- 
ter q that modifies [step (iii)] , so that with probability q 
the new neighbor is selected among the neighbors of the 
neighbors; otherwise with probability 1 — q the random 
neighbor is chosen. We find that, while most of our re- 
sults previously discussed are qualitatively independent 
of the value of q, the clustering coefficient reaches a very 
large value even for a small value of q. For instance, 
a slightly 1% (q — 0.01) of local neighbor selection is 
enough to increase c a hundred times, being the cluster- 
ing largest for a slow evolution of the network (p < 1). In 
addition, the clustering coefficient decreases slightly with 
system size, an indication of a decay slower than the iV -1 
decay expected for random graphs. All together our re- 
sults indicate that local neighbor selection is needed in 
order to generate a small world network. 

It is worth noting that an evolutionary model based on 
the PD game with a more complex strategy representa- 
tion also shows, in the absence of local neighbor selection, 
that the increase of the clustering coefficient can be re- 
lated to the change of the degree distribution 0. In 
contrast with Ref. |l5j |. we don't observe a power law 
degree distribution. 

Dynamics: global avalanches and network stability. - 
The hierarchical structure of the network is of fundamen- 
tal importance to the dynamics on the system. A closer 
look at the the evolution towards a stationary state indi- 
cates the presence of avalanches (Fig. |5J| . The transient 
dynamics is characterized in general by large oscillations 
in pc(t)- When the payoff of any D element increases 
above the upper limit of Dq. an avalanche towards 
defection is triggered. This D element will be imitated by 
C neighbors, and each will initiate an avalanche of repli- 
cation of D state through all those elements connected by 
the imitation network. During the avalanche, recovery of 
cooperation is possible through those satisfied Coopera- 
tors, which re-build the hierarchical topology • 

The description in terms of the imitation network also 
indicates the vulnerability of the structure to stochastic 
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FIG. 5: Time evolution of pc- The evolution starts in a fixed 
random network (p — up to time t — 200 when network 
dynamics is switched on, so that p = 1 for t > 200. At time 
t — 500 the state of the node with largest payoff is forced 
from C to D. Parameter b = 1.7. 

fluctuations. Figure illustrates the sensitivity of the 
stationary network structure to perturbations acting on 
the highly connected nodes, which reflects their key role 
in sustaining cooperation. At time t = 500, the most 
connected node is externally forced to change state from 
C to D, triggering an avalanche. Notice the large os- 
cillations in pc, reproducing the transient dynamics in 
which the system searches for a new stationary globally 
cooperative structure. 

Conclusion.- We have addressed the general question 
of network formation from the perspective of co-evolution 
between the dynamics of the elements' state and the in- 
teractions network. Our model of cooperation with net- 
work plasticity leads to hierarchical topologies [l^, the 



emergence of global cascades 0, |2(| and vulnerability 
to attacks acting on specific targets |2l|. The hierarchi- 
cal interaction network is reached as a stationary network 
starting form a random network of interactions. The net- 
work appears structured from a few highly connected el- 
ements easily identified through the imitation network. 
Such network has the characteristics of a small world 
when a mechanism of local neighbor selection is intro- 
duced in the adaptive dynamics of the network. The hi- 
erarchical structure supports a stationary highly cooper- 
ative state for general situations in which for a fixed net- 
work the system would not settle in a stationary state and 
in which the cooperation level would be much smaller. 
The stability of the network is very sensitive to changes in 
state of the few highly connected nodes: external pertur- 
bations acting on these nodes, trigger global avalanches 
leading to transient dynamics in which the network com- 
pletely reorganizes itself searching for a new highly coop- 
erative stationary state. Future work should explore the 
robustness of these results in slightly different settings. 
For instance we have checked that the same qualitative 
results are obtained with asynchronous update regard- 
ing Figure 1, and that adding continuous noise weakens 
the cooperative phase by the spontaneous occurrence of 
avalanches. Work along these lines is in progress. 
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